On the higher-order derivatives of spectral functions

In this paper we are interested in the higher-order derivatives of functions of the eigenvalues of symmetric matrices with respect to the matrix argument. We describe the formula for the k-th derivative of such functions in two general cases. The first case concerns the derivatives of the composition of an arbitrary (not necessarily symmetric) k-times differentiable function with the eigenvalues of symmetric matrices at a symmetric matrix with distinct eigenvalues. The second case describes the derivatives of the composition of a k-times differentiable separable symmetric function with the eigenvalues of symmetric matrices at an arbitrary symmetric matrix. We show that the formula significantly simplifies when the separable symmetric function is k-times continuously differentiable. As an application of the developed techniques, we re-derive the formula for the Hessian of a general spectral function at an arbitrary symmetric matrix. The new tools lead to a shorter, cleaner derivation than the original one in [16]. To make the exposition as self contained as possible, we have included the necessary background results and definitions. The proofs of the intermediate technical results are collected in the appendices.